Statistical approaches to image and scene manipulation

The field of visual perception aims to study the human visual system. It is recognized that in order to understand a system, it is important to understand its input. This has led to the sub-field of natural image statistics, which tries to understand images in statistical terms. Images’ statistical properties are usually studied by collecting a number of such images in an ensemble and computing first, second or higher order statistics on them. One of the most well studied natural image statistics is that of the power spectrum, a second order statistic. By computing the Fourier transform of an image and multiplying each element of the transform by its complex conjugate, the power spectrum is obtained. Averaging over all directions gives power as function of frequency. For most natural images, plots of this quantity result in straight lines with a slope of around 1/f, where f is the spatial frequency (usually measured in cycles per image) [2–4]. This is a special property of natural images, which is generally not obtained for random images (such as for example random noise images, which would produce a flat power spectrum). The 1/f spectral slope of natural images means that equal power is encoded in each frequency band. It also implies that natural images are statistically scale-invariant. While the reason for this to occur is still being debated (e.g. [1]), these results are important for graphics applications. It has been shown that certain image interpretation tasks are negatively affected when the spectral slope deviates too much from −2 [6, 7]. It is therefore argued that the human visual system expects to see images that conform to this statistic. As graphics applications produce input to the human visual system, it makes sense to optimize this input to be as easy to interpret as possible. Conforming to second order statistics is one step in that direction. Although individual images may have spectral slopes that are somewhat different from 1/f, the variation between images is relatively small. Hence, the power spectrum provides a simple and elegant means to assess the quality of an image: if the power spectrum does not yield a straight line and if the slope of this line deviates much from −2, then the image is less realistic in a statistical sense. Recent research has shown that the power spectrum is sensitive to modeling, but is quite insensitive to variations in rendering. Different levels of image compression ∗University of Utah, [email protected]